Verification of Programs Manipulating Complex Dynamic Data Structures

We develop a verification method based on a novel use of tree automata to represent heap configurations to allow verification of important properties—such as no nullpointer dereferences, absence of memory leaks, etc.—for programs manipulating complex dynamically linked data structures. In our approach, a heap is split into several “separated” parts such that each of them can be represented by a tree automaton. The automata can refer to each other allowing the different parts of the heaps to mutually refer to their boundaries. Moreover, we allow for a hierarchical representation of heaps by allowing alphabets of the tree automata to contain other, nested tree automata. Program instructions can be easily encoded as operations on our representation structure. This allows verification of programs based on symbolic state-space exploration together with refinable abstraction within the so-called abstract regular tree model checking. A motivation for the approach is to combine advantages of automata-based approaches (higher generality and flexibility of the abstraction) with some advantages of separation-logic-based approaches (efficiency).